Papers with real-world requirements
SMART: Scalable Multilingual Approach for a Robust TOD System (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Existing TOD frameworks face significant challenges in handling unstructured information, providing multilingual support, and engaging proactively. |
| Approach: | They propose a novel TOD framework that combines traditional pipeline elements with modern agent-based approaches and features a simplified dialogue state, intelligent clarification mechanisms, and a unified natural language generation component that eliminates response redundancy. |
| Outcome: | The proposed framework outperforms baseline systems across key metrics and integrates in an e-commerce store. |
Real-World Compositional Generalization with Disentangled Sequence-to-Sequence Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to compositional generalization have been designed with semantic parsing in mind. |
| Approach: | They propose a disentangled sequence-to-sequence model which encourages more disentanglement and improves its compute and memory efficiency. |
| Outcome: | The proposed model improves generalization performance across existing tasks and datasets and a new machine translation benchmark. |
MobileBench-OL: A Comprehensive Chinese Benchmark for Evaluating Mobile GUI Agents in Real-World Environment (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in mobile Graphical User Interface (GUI) agents highlight the growing need for comprehensive evaluation benchmarks. |
| Approach: | They propose an online benchmark with 1080 tasks from 80 Chinese apps that measures task execution, complex reasoning, noise robustness and auto-eval framework with a reset mechanism. |
| Outcome: | The proposed benchmark measures task execution, complex reasoning, and noise robustness of agents by including 5 subsets, which set multiple evaluation dimensions. |
SpeakerSleuth: Can Large Audio-Language Models Judge Speaker Consistency across Multi-turn Dialogues? (2026.acl-long)
Copied to clipboard
| Challenge: | Large Audio-Language Models (LALMs) are a popular approach for evaluating speech quality, yet their ability to assess speaker consistency across multi-turn dialogues remains unexplored. |
| Approach: | They construct 1,818 human-verified evaluation instances across four datasets spanning synthetic and real speech, with controlled acoustic difficulty. |
| Outcome: | The proposed model performs better in comparing and ranking acoustic variants, demonstrating inherent acustic discrimination capabilities. |
DecompileBench: A Comprehensive Benchmark for Evaluating Decompilers in Real-World Scenarios (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches focus on syntactic correctness through synthetic micro-benchmarks or subjective human ratings, despite semantic fidelity and usability. |
| Approach: | They propose a framework that enables effective evaluation of decompilers in reverse engineering workflows . they compare six industrial-strength decompils and six recent LLM-powered approaches . |
| Outcome: | The proposed framework outperforms commercial tools in code understandability despite lower functionality correctness . it shows that it can transform human-centric reverse engineering workflows . |